The Basic AI Drives
نویسنده
چکیده
One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation and to improve themselves. We then show that self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional systems which will want to modify their utility functions. We next discuss the drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.
منابع مشابه
Omohundro’s “Basic AI Drives” and Catastrophic Risks
Shulman, Carl. 2010. Omohundro's " Basic AI Drives " and Catastrophic Risks.
متن کاملImproved Direct Torque Control for Induction Machine Drives Based on Fuzzy Sector Theory
Here, a new fuzzy direct torque control algorithm for induction motors is proposed. As in the classical direct torque control, the inverter gate control signals directly come from the optimum switching voltage vector look-up table, the best voltage space vector selection is a key factor to obtain minimum torque and flux ripples. In the proposed approach, the best voltage space vector is sel...
متن کاملFormalizing Convergent Instrumental Goals
Omohundro has argued that sufficiently advanced AI systems of any design would, by default, have incentives to pursue a number of instrumentally useful subgoals, such as acquiring more computing power and amassing many resources. Omohundro refers to these as “basic AI drives,” and he, along with Bostrom and others, has argued that this means great care must be taken when designing powerful auto...
متن کاملArtificial intelligence & genetics
Artificial intelligence (AI) is the development of computer systems that are able to perform tasks that normally require human intelligence. Artificial intelligence (AI) is a wide-ranging tool that enables people to rethink how we integrate information, analyze data, and use the resulting insights to improve decision making—and already it is transforming every walk of life. AI has application...
متن کاملApplication of Speed Estimation Techniques for Induction Motor Drives in Electric Traction Industries and vehicles
Induction motors are the most commonly used in the traction industries and electric vehicles, due to their low primary cost, low maintenance costs, and good performance. Speed identification is needed for the induction motor drives. However, using of speed sensors in the induction motor drives is associated with problems such as, extra cost, reduced reliability, added mounting space, etc.. Ther...
متن کامل